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Abstract 

Wc consider a control problem where the state must approach asymp- 
totically a target C while paying an integral cost with a non-negative 
Lagrangian I. The dynamics / is just continuous, and no assumptions 
arc; made on the zero level set of the Lagrangian I. Through an inequal- 
ity involving a positive number po and a Minimum Restraint Function 
U = U{x) -a special type of Control Lyapunov Function- we provide a 
condition implying that (i) the system is asymptotically controllable, and 
(ii) the value function is bounded by U /po. The result has significant con- 
sequences for the uniqueness issue of the corresponding Hamilton-Jacobi 
equation. Furthermore it may be regarded as a first step in the direction 
of a feedback construction. 
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1 Introduction 



Let C C M" be a closed subset, which wih be cahed the target, and let 
denote its complement. We consider the value function 



V(x) = inf 

Jz,a= / l{z{t),a{t))dt 



(1) 



(2) 



for trajectory-control pairs {z,a) : [0, Tj[:— )■ x A subject to 

= /(z(t),a(t)) z{0) = x 

lim^^j,- d{z{t),C) = 0, 

where d denotes the Euclidean distance. 

The crucial assumption of the paper will be the sign condition 

l{x,a)>0 V(a;, a) G X A. (3) 



In its stronger form, the main result of the paper (Theorem 1.1) reads as 
follows: 

Let pQ be a positive real number and let U = U{x) be a proper, positive 
definite, semiconcave function such that, for every x £ C^, one has 

H{x,Po,D*U{x))<^.^ (4) 

Then the system is globally asymptotically controllable and the value function 
V verifies the inequality 

V{x) < U{x)/Po, 

for all X G C^. 

In (41), H denotes the natural (minimized) Hamiltonian of the system, 
nameljj^ 

H{x,po,p) = M^(^{po,p) , (l{x,a)J{x,a)^'^, (5) 



^The notation H{x, po, D*U{x)) < means that H{x,po,p) < Vp G D*U[x). 

■^The fact that the state variable is n-dimensional while the adjoint variable (po,p) is 
(n + l)-dimensional is due to the presence of a hidden state variable 20, namely the one 
verifying the differential equation io(i) = ^^C^))- 
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while D* is the generahzed differential operator called limiting gradient 



(see Def. 1.4) 



A function U as above is here called a Minimum Restraint Function (Def. 

ri]). 

Let us make clear that condition Q is not a mere application of the 
usual (first order) asymptotic global controllability condition to the enlarged 
dynamics obtained by adding the equation zq = l{z,a), with the enlarged 
target [0, +oo[xC. Actually, the known conditions on the existence of a Con- 
trol Lyapunov Function to characterize global asymptotic controllability would 
provide no information on the value of the minimum Let us also anticipate 
that our result is valid under general hypotheses on / and / that do not guar- 
antee neither uniqueness nor bounded length in finite time of the trajectories 
(see next subsection). 

Investigations on this kind of value functions have been pursued in several 
papers, mainly from the point of view of the corresponding Hamilton-Jacobi 
equation. Indeed the existence of pairs (x, a) such that l{x,a) = raises vari- 
ous non trivial problems about uniqueness. A likely incomplete bibliography, 
also containing applications (for instance, the Fiiller and shape-from-shading 
problems), includes |BCDj . P, HE], [CSic] . [Sor2] . [M], |M3> and the references 
therein. 

Actually, besides displaying an obvious control theoretical meaning (see 



also Remark 1.4), our result provides a sufficient condition for the value func- 
tion to be continuous on the target's boundary. This continuity property is 
crucial to recover uniqueness of the corresponding boundary value problem 



(see Remark 1.3). 



We conclude this informal presentation by observing that if some ?? > 
existed such that l{x,a) > r/ V(x,a) G K" x A, then the problem could 
be easily reduced to an actual optimal time problem by just utilizing the 
reparameterized dynamics 

dy ^ f{y,a) 
dr l{y-,(y) 

Indeed, after the (bi-Lipschitz) time-parameter change r(t) = /q I dt the La- 
grangian turns out to be transformed into the constant value 1. Yet, if only 
the weaker sign condition (jsj) is assumed, a direct approach based on such a 
reparameterization cannot be adopted. 



^ Notice also that W{xo,x) = po xo + U (x), namely the function one differentiates in the 
extended space, is not proper. 
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The paper is organized as follows. In the remaining part of the present 



section we state rigorously the main result of the paper (Theorem 1.1) and 
provide some basic definitions. In Section [2] we sketch the main result's proof 
by heuristic arguments and give a geometrical description of the thesis. Section 
[2] ends with some examples. Section [3] is the longest one, and is entirely devoted 
to the proof of Theorem |1.1[ while some technical results are proved in Section 
|4) In Section [5] we make some remarks on the meaning of assumption ( [4| ) as a 
viscosity supersolution condition. 

1.1 Precise statement of the main result 

Our main technical assumptions are: 

(i) for given positive integers n, m, the controls a(-) take values in a compact 
set A C M"* and are Borel-measurable, while the state values z{t), x 
range over M"; 

(ii) the target C is closed and has compact boundary; 

(in) the augmented vector field (/,/) is a continuous function on x A. 

In particular, for any given control a(-) and initial condition 2;(0) = x, the 
Cauchy problem associated to the differential equation ([2]) may have multiple 
Caratheodory solutions. Moreover, the latter may have unbounded velocity 
near C, since / itself is not assumed to be neither Lipschitz nor bounded near 
C. Actually it may well happen that approaching trajectories fail to reach the 



target even when Tz < +00 (see Example 2.3) 



Let us introduce the notion of Minimum Restraint Function. 

Definition 1.1 We say that a continuous function U : ^ TR is a Minimum 
Restraint Function, in short, a (MRF), if U is locally semiconcave, positive 
definite, and proper on C^, and, moreover, there exists po >0 such that 

H{x,po,D*U{x)) <0 (6) 

holds true for all x £ 



We refer to Subsection 1.2 for the definitions of limtting gradient, D*U, and oi proper, 



positive definite and semiconcave function. 



4 



Theorem 1.1 Let a Minimum Restraint Function U exist. Then: 
(i) the system ^ is globally asymptotically controllabl^ to C; 
(ii) if U is a Minimum Restraint Function with po > 0, then 

Vix) < ^ Vx G (7) 
Po 



Remark 1.1 Petrov-like inequalities (see Example 2.1) are included in con- 
dition ([6]). However, let us point out that they only concern the case where 
/ = 1 and the dynamics is bounded near the target, which implies that optimal 
(or quasi-optimal) trajectories take a finite time to reach the target. Instead 
when the Lagrangian is just non-negative, condition ([g]) does not force optimal 
trajectories to approach the target in finite time. 



Remark 1.2 Because of the sign assumption ([3|, a Minimum Restraint Func- 
tion, (MRF), is in particular a Control Lyapunov Function, (CLF). Hence, as a 
byproduct of statement (i) in Theorem |1.1| we get an extension to merely con- 
tinuous, unbounded dynamics of the results concerning the relation between 
Control Lyapunov Functions and asymptotic controllability (see e.g. |S2j). 



Remark 1.3 Because of the bound ([7]), the value function V turns out to be 
continuous on dC. Actually, this is a theoretical motivation for a result like 
Theorem |l.l[ in that the continuity of the value function on the target's bound- 
ary is essential to establish comparison, uniqueness, and robustness properties 
for the associated Hamilton- Jacobi-Bellman equation (see |MSj . |Sorlj . and 
(Ml). 

Remark 1.4 Our investigation might be useful also for feedback control. In- 
deed, on the one hand the uniqueness of the H-J equation is an obvious ingre- 
dient in a feedback-oriented construction. On the other hand, a (MRF) U can 
be possibly exploited in order to build a "safe" feedback law for problem ([T]) 
(see e.g. jCLSSj and jABj for general reference to feedback stabilization). 



Remark 1.5 It is easy to adapt Theorem 1.1 to the case when the state space 



is an open set Q C H*^, O D C. In fact, the thesis keeps unchanged as soon 



'See Definition 1.5 
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as one requires the (MRF) U : 17 \ C — t- H to verify all the assumptions in 
Definition 1.1 in 0, plus the following one: 

3Uo G]0, +oo] : lim U{x) = Uq ^xq G 90; U{x) < Uq Vx G J] \ C. 

x—^xo, x(^Q 

1.2 Basic definitions 

For the reader convenience, some classical concepts, like (GAC), and a few 
technical definitions (part of which have already been used in Theorem |1.1| 
above) are here recalled. 

Definition 1.2 (Positive definiteness). A continuous function U : ^TR is 
said positive definite on if U{x) > Vx G and U{x) = Vx G dC. 
Moreover U is called proper on if U~^{K) is compact as soon as K C 
[0, +oo[ is compact. 

Definition 1.3 (Semiconcavity). Let 17 C M" be an open set, and let F -.Vt ^ 

IR 6e a continuous function. Fis said to be locally semiconcave on 17 if for any 
point X G 17 there exist R > and p > such that 

F{zi) + F{z2) - 2F (^^^] < R\zi - Z2\^ Vzi, Z2 G B„(x, p). 



Let us remind that locally semiconcave functions are locally Lipschitz. Actu- 
ally, they are twice differentiable almost everywhere (see e.g. |CB j ) . 

Definition 1.4 (Limiting gradient). Let 17 C H" be an open set, and let 
F : 17 — 7- IR 6e a locally Lipschitz function. For every x £ ^1 let us set 

D*F{x) = G : w = limVF(xfc), Xk G DIFF{F)\{x}, limxfc = x| 

where V denotes the classical gradient operator and DIFF(F) is the set of 
differentiability points of F. D*F{x) is called the set of limiting gradients of 
F at X. 

For every x G 17, D*F(x) is a nonempty, compact subset of M"" (more precisely, 
of the cotangent space T'*17). Notice that, in general, D*F{x) is not convex]^ 

^Actually its convexification coincides with the Clarke's generalized gradient. 
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To give the notion of global asymptotic controllability we need to recall 
the concept of function belonging to ICC: these are continuous functions /3 : 
[0, +oo[x[0,+oo[-;> [0,+oo[ such that: (1) /3{0,t) = and (3{-,t) is strictly 
increasing and unbounded for each t > 0; (2) /3(r, •) is decreasing for each 
r > 0; (3) (3{r,t) — as t — +oo for each r > 0. For brevity, let us use the 
notation d(x) in place of d(x, C). 

Definition 1.5 [^GAC) The system ^ is globally asymptotically control- 
lable to C -shortly, ^ is (GAG) to provided there is a function f3 G ICC 
such that, for each initial state x G C^, there exists an admissible trajectory- 
control pair {z, a) : [0, +oo[— )■ x A that verifies 

d{z{t)) < /3(d(x),t) yt G [0,+oo[. (8) 



In Theorem |l.l[ s proof we shall make use of the notions of partition of an 
interval and of its diameter. To avoid vagueness let us state the precise meaning 
we attach to these terms. 

Definition 1.6 Let us consider an interval [0,b[, b g]0,+oo]. A partition 
of [0,b[ is a sequence vr = (t^) such that t^ = 0, t^~^ < t^ Vj > I, and 
either limj_^+oo t^ = b or there exists some G IN such that t^^ = b. In the 
latter case, we say that ir is a finite partition of [0, b[. The number diam{Tr) = 
supjj. p<b}{'^'^ ~ is called the diameter of the sequence vr. 



2 Heuristics of the proof and some examples 

2.1 A one dimensional differential inequality 

As in the case of asymptotic controllability, the underlying idea of Theorem 



1.1 relies on a one-dimensional argument involving a differential inequality. 
To express this issue, let us assume some simplifying facts. Let us begin with 
making the hypothesis that U is of class C^, so that assumption (|6|) reads 



inf ( {Po, VU{x)) , ( l{x, a), f{x, a) ) ) < Vx G C^. (9) 



^We call a real map (f> decreasing [increasing], if (f>{ri) > cj>{r2) [4>{ri) < 4'{r2) ] as soon as 
ri < r-2 and strictly decreasing [increasing], if the inequality is always strict. 

^ To be precise, we are considering a slight variation of the standard notion of (GAC) to 
C, which would require C to be weakly invariant with respect to the control dynamics, since 
we are interested in the behavior of any admissible trajectory z just for t £ [0, [■ Therefore, 
we fix an arbitrary z £ dC and, when < +cx3, we prolong z to [0, +cx3[ by setting z{t) — z 
for all t > T^. 
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Let us also assume that there exists a continuous selection 

a{x) G A{x) = |a G A : ^(po, V?7(x)) , (l{x, a),f{x, a)) ^ < o} 

so that ^ yields 

'iPo, VU{x)) , {l{x, a{x)), fix, a(x))) ) < Vx G 



(10) 



for some po > 0- 

Let us consider a solution of the augmented Cauchy problem 

io = l{z,a{z)) 
z = f{z,a{z)) 
[ (zo,^)(0) = (0,x) 



(11) 



and let us set 



Then, by (10), 



m = u{z{t)). 

i{t)+PoUt) = {VU{z{t)),i{t))+Poio{t) = 

{VU{z{t)) , f{z{t), a{z{t))) + Po io(t) < 



(12) 



Notice that ^(0) = U{x) > and zo{0) = 0. The rough idea of the proof (to be 
sharpened through suitable nonsmoothness' and o.d.e.'s arguments) amounts 
to show that: 

(A) ^(t) is defined, strictly decreasing and tends to zero in a possibly un- 
bounded interval [0, r[ : this means that lim^^T d(2;(i), C) = 0, which 
coincides with part (i) of the thesis of Theorem |1.1[ 

(B) If po > 0, the rate of growth of the (non negative and increasing) map 



zo(t) is bounded by — this implies that 

Po 



V{x) < lim zo{t) < — ^(0) - lim ^(t) 



t-5>T 



Po 



U{x) 



Po 



which coincides with the statement (ii) of Theorem [lT] 
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Let us observe, however, that inequahty (12) is not enough to prove (A). 
In fact, ^(•) could well decrease asymptotically to a value ^ > 0. To show that 
^(•) actually converges to zero, one deduces the differential inequality 



(13) 



from ( 12 ), m(-) being a suitable positive, strictly increasing function on ]0, +oo[ 



This is, in fact, the essential content of Proposition 3.1 where (13) is replaced 



by the nonsmooth relation (18). The other ingredient of the proof is Proposi- 



tion [3^ where nonsmooth analysis techniques are applied to show that things 
actually work even without the simplifying regularity we are assuming here. 
In particular, when pQ > one can emulate the above (B) and get the bound 
on the value function. 



2.2 A geometrical insight 

A further interpretation of the result in part (ii) of Theorem |1.1| in the case 
po > 0, is provided by the following "geometrical" description of the above 
heuristic arguments. To begin with, notice that the (1 + n)-dimensional target 
C = [0, +oo[xC has no longer compact boundary. Therefore, no proper (CLF) 
can exist. Actually, a (MRF) U, when considered as a function on [0, +oo[x]R", 
is not proper. On the other hand, let us consider the map 

W{xo,x) = U{x) +poxo. 

The key point which makes Theorem 1 1 . 1 1 work relies on the following three 
facts: 

1) W is proper in [0, +oo[x]R"; 

2) the inequality 

H{x,Dl^^^^^^W{xo,x))<0, (14) 

which coincides with inequality ([6]), says that, for every x ^ C^, there 
exist trajectories of the augmented system {zo,z) = {l,f) starting from 
{0,x) and remaining inside the x-influence set 

{(xo,x) G [0,+oo[x]R" : W{xo,x) < ^/(x)}; 

3) the level sets of W intersect the extended target C. 
The situation is illustrated in Fig.l. 
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\p^* U(x)/p=] W(y^y) = W(O^) {=U(xn 
^'V(x)< y=U(x)/p„ 




C = [0,+colxC 



(0,x) (0,y) ^ 

W(x^x) = W{0;i) l=U{x)] 

^H^^^^l x^mfluence set 



A taaectory 



Figure 1: The level sets of W intersect the extended target C 



2.3 Some examples 

Example 2.1 A prototype of (MRF) is a map of the form U{x) = $ o d(x), 
where <I> : [0, +oo[— )• IR is a continuous map such that '&(0) = and its 
restriction to ]0, +cxd[ is a strictly increasing C-'^-diffeomorphism. 
In particular, in the minimum time problem, where 1=1, the inequality ([g]) 
includes the following weak Petrov condition (see e.g. |Slj . |CS| . and |BP| ): 

(P) there exist 6 > and a continuous, increasing map /i : [0,6] — [0,+oo[ 

verifying ^(0) = 0, fi{p) > for p > 0, / — — < +oo, and such that, 

Jo f^[P) 

Vx G with d(x) < 5, one has 

mUD*d{x),f{x,a)) < -A^(d(x)). (15) 

Indeed, if we set <I>(r) = Jqi^ for all r G [0,S] and choose an arbitrary 
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Po £ [0, 1[, we can write (15) as[^ 

H{x,Po, D*{^ o d)(x)) = inf {Po + {D*{<!> o d){x)J{x, a))} <-{l-Po)< 0. 

aeA 

In particular, whenever $ is linear one recovers the classical Petrov condition. 

Example 2.2 Let r, s be arbitrary real numbers and let ^l^l{x), ip2{x) be con- 
tinuous functions such that V'i(^) ^ -^i ^-nd < il^2ix) < M2 for some Mi, 
M2 > 0. Consider the target C = {0}, the control dynamics 

z = aipi{z)\z\'^ aG{ — 1,1} 

and the Lagrangian 

l{z)=Mz)\z\'. 

Notice that / can be zero on an arbitrary subset of its domain. 
Let s, r verify 

s-r>-l (16) 



and consider the map 



M2 



U(x) = ; r \X 

^ ' Mi{s-r + l) ' 



Observe that U is proper, positive definite and semiconcave on IR \ {0} 
actually, U G C°°(iR. \ {0}). Moreover, for any po > 0, 



< 



H{x,po,p) = inf„g{_i_i} ^poiJ2ix) \x\^ + aptpi{x) 
P0M2 \x\^ - Ml \p\\x\\ 

Since 

D*U{x) = {VU{x)} = |sign(x) Vx / 0, 

one obtains 

H{x,po,D*U{x)) <0 Vx^O, 

as soon as po g]0, 1[. 



^This equivalence follows from the straightforward set identity o d)(a;) 
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By applying Theorem 1.1 we get that the value function V — namely, the 
minimum cost to attain the target (possibly in infinite time) — , verifies 



V(x) < 



M2 



-r+l 



Mipo(s -r + l) 
for all £]0, 1[, which implies 

M2 



V(x) < 



Ml (s -r + l) 



X 



s— r+l 



Vx G M. 



Notice that (16) is crucial. Indeed, if s — r < —1, a nonsingular (MRF) 
may fail to exists. In fact, consider the trivial case when -01 = 'i/'2 = 1. If J7 
were a nonsingular (MRF), for almost every x g]0, 1] we should have 

\VU{x)\ > Pox'^' >pox-^ 

for some pQ > 0, which clearly prevents U to be positive definite. 



In Example 2.2 the dynamics may happen to be unbounded (precisely, 
when r < 0). However, when the time to approach the target happens to be 
finite, the trajectory's interval can be prolonged to the closed interval [0, T^], 
so that z(Tz) G C. Let us remark that this is due to the one-dimensionality of 
the state space. In fact in the next example we see, among other things, that 
there is a connected component of the target that can be approached in finite 
time while it cannot be reached. 
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Example 2.3 Let us set 

c = Ci u C2 c Il^ 

where 

Cl = {xG]R^ |x| < 1}, C2 = {xGlR^ \x\>4}, 
and let us consider the control dynamics 

M-z 

z = -—, — az 

^"^Incidentally, this is the reason why we have adopted a notion of (GAC) slightly more 
general than the usual one. 
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with the control a taking values in {—1, 1} and M = ^ ^ . FinaUy let us 

consider the Lagrangian 

r {\z\ - l)^k{z) if 1 < |z| < 2 

l{x) = < {3-\z\ fk{z) if 2 < y < 3 , 

1^ if 3 < |a;| < 4 

where : — )• ]R is any continuous function verifying < k{z) < 1 Vz G C^. 
Let us begin with showing that the target's component Ci can be indefinitely 
approached in finite time but it cannot be reached. Indeed, in polar coordinates 
z = yoe*^ the control equation becomes 

9 = —ap 

Let p G]1,4[, 9 £ [0,27r[. Choose z = pe'^ G C^, a = 1, and consider the 
solution of the first equation: p : [0,ln/)[— t- IR, p{t) = pe^*. The time this 
trajectory takes to approach C is equal to In p. Furthermore, 



It follows that 



lim {p{t),9{t)) = (l,+oo). 

t—^ln p 



lim d{z{t)) = 

i— s>ln p 



but z{t) has no limit as t — t- In p . Namely, the trajectory spirals faster and 
faster around Ci while approaching it. 

On the other hand it is trivial to see that the target's component C2 can 
be reached in finite time by implementing the constant control a = —1. Lastly, 
setting for every e > 0, 



2e+^^^ ifl<U|<2 



e{A-\z\) + ^ — ^ if2<|z|<3 



. e(4- |z|) if 3 < |z| < 4 

one can easily check that as soon as e > f/^ is a Minimum Restraint Function 
with po = 1. Therefore, if V(x) denotes the value function of the problem, by 
Theorem 1 1.1 1 we get V{z) < U^{z) for every e > 0, so that 

V{z) < Uo{z). (17) 
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Notice that, in agreement with the definition of (MRF), the functions 
Uc are not smooth at their maximum points. Moreover, the inequality (17) is 
optimal in the ring R = {z \ 3 < \z\ < 4}, in that Uq = 0, and hence V = 0, 
on R. 



3 Proof of the main result 

3.1 Preliminary results 

The proof of Theorem 1 1 . 1 1 relies on Propositions 3.1 and 3.2 below. In order to 
retain clarity in the main proof's argument, we postpone the proofs of these 
technical results to Section HI 

Proposition 3.1 Let U be a (MRF) and let pQ > make ^ hold true, i.e. 

H{x,po,D*Uix)) <0 VxgC^ 

Then for every a > the map U verifies also the differential inequality 

H{x,po,D*U{x)) < -m{U{x)) Vx G U-^{]0,a[), (18) 

where m : [0,+oo[— )• TR is a suitable continuous, strictly increasing function 
verifying m{r) > Vr > 0. 

Remark 3.1 If we replace condition Q, namely 

3po: H{x,po,p)<0 VxgC^ \/p£D*U{x), 

with the stronger assumption 

3po : VM > 0, sup H{x,po,p)<0, 

d{x)>M; p&D*U{x) 



then by slight changes in the proof of Proposition 3.1 below, one can prove 
that there exists a continuous, strictly increasing function m : [0,+oo[— >• IR, 
independent of a, verifying m(r) > Vr > 0, such that ( [l8| ) holds for all 
X G C. 
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Proposition 3.2 Let U be a (MRF) and let o" > 0. Let m he defined as in 



Proposition 3.1. Fix e > and fi, fx such that < jl < fl < a. Then there 
is some 6 > such that, for every S' G]0,5[ and for each x £ verifying 
U{x) = p,, for a suitable t > one can construct a trajectory-control pair 

{z,a) : [0,t\ U-^{[fL,fi]) X A, z{0) = x, 

and a finite partition vr = {t^, . . . , t"} of [0, i\ such that diam{'K) < 5' and with 
the following properties: 

(a) U{z{i)) = fi< U{z{t)) < U{z{0)) = U{x) = /i Vt G [0,t[; 

(b) for every j £ {1, . . . , n} and Vt G [t-'^^, t^[, 

U{z{t^)) < U{z{t)) < U{z{t^-^))- (19) 

(c) for every j G {1, . . . , n} and Vt G [t-^^^ , t-'], 

U{z{t)) - U{z{t^-')) + ^ l{z{t),a{t)) dt < 

-^^!Um{U{z{t)))dt. 



(20) 



3.2 Proof of Theorem ITTTl 

In the sequel, given a constant /i > 0, for any continuous path y : [r, +oo[— )■ M" 
with U{y{T)) > fi, we define the time to reach the enlarged target f7^-'^([0, ^]) 
as 

Ty^ = inf{r > r : U{y{r)) < ^} (21) 
(7/ = +00 if U{y{r)) > /i for aU r > r). 



Let cr > be a positive constant and let m be defined as in Proposition 3.1 
Fix e > and let (i/^,) c]0, 1] be a sequence such that 1 = > ui > 1^2 > ■ ■ ■ 
and hmfc_!.oo J^fc = 0. Assume that x G U~^{]0,a[) and set 

Aifc = T^kU{x) \/k > 0. 



We are going to exploit Proposition 3.2 in order to build the trajectory- 
control pair 

(z, a) : [0, C x A 
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by concatenation, 

{z{t),a{t)) = {zk{t),ak{t)) \/t e [tk-i,tk[, yk > 1. 

Step k = 1. Let us begin by constructing {zi, ai). Setting jl = /Uq, fi = ni, 
let (2:1,01) : [0 , t\ — >• U~^{[fii, fio]) X ^ be a trajectory built according to 



Proposition 
that h = Ti 



3.2 



such that zi{0) = x. We set io = and ti = i and we observe 



3.2 



zi , in view of (a) in Proposition 
Step k > \. Let us proceed by defining (2:^,0^) for k > 1. Setting Ji = 
//fc_i, fi = fik, let (%.,Q;fc)j_|0,f] -)> C/"H[Aifc, Aifc-i]) x A be a trajectory built 
according to Proposition 3.2 such that ifc(O) = Zk-i{tk-i). We set tk = tfe-i +t 
and {zk,ak){t) = {zk,ak){t - tk-i) Mt G [tfc_i,tfc]. We observe that = T^^ , 
still in view of (a) in Proposition |3.2[ 

The concatenation procedure is concluded as soon as we set t = Wm.^^^ t^- 
Notice that it may well happen that i = +00. We claim that 



lim d{z{t)) = 0. 



(22) 



For every A; > 1, let us apply Proposition 3.2, which yields the existence of a 
finite partition vr^ = {t^, . . . , t^''} of [0, tjt — ifc_i] such that, setting. 



tl = tk-i + il Vj G {0, . . . , nfc}. 



one has z(0) (= zi{Q)) = x, and, for every k > 1: 
(a)k Zk+iitk) = Zkitk), U{zk{tk-i)) = Hk~i; 
{h)k for ah j G {1, . . . ,nfc}, 

u{zk{ti)) < u{zk{t)) < u{zk{ti-^)) < u{x) yt G [tr\ti[; 



(c)fe for ah j G {1, . . . ,nfc}. 



U{zk{t)) - Uizkitl')) + ^ l{zk{t),ak{t)) dt 



< 



^jUm{U(zkmdt yt€[ti-\ti] 



Indeed, in view of point (b)^ above, (22) is equivalent to 

lim d{zkitk)) = 0. 



(23) 



On the other hand, since U is proper and positive definite, (23) is a straight- 
forward consequence of 



lim U{zk{tk)) 

fc— >-oo 



lim z/fc U{x) 
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Therefore (22) is verified. 



Notice tliat (h)^ implies also that 

U{z{t)) <U{x) <a VtG[0,t[. 



(24) 



In order to conclude the proof that the system is (GAC) to C (part (i) 
of the theorem), we have to establish the existence of a ICC function /3 as in 
Definition 11.51 

Let k >1. From condition (c)^ and in view of the definition of {zk,ak), 
we have Vt G 

U{zk{t)) - U{zk{tk-i)) = [U{zk{t)) - U{zk{i^-'))] + [t/(zfc(tr')) - U{zk{i-''))] 
+ ... + [U{zk{tl))-U{zk{tl))]< 

-ifl /i„, l{z,{T),a,{T)) dr - ^ m{Uiz,{r))) dr, 
which implies that Vt € [0, 

1 



U{z{t))-U{x) < 



/ liz{r),air))dT-— I m{U{z{T))) dr. (25) 



Being / > 0, in particular we have 



U{z{t)) - U{x) < 



jlm{U{z{T))dT 



e + l 



Since m.i\i{m{U {z{t)) : r G [0, = m(C/(z(t;[)), we get 



£ + 1 



k\- 



(26) 



Observe that the function rh : [0, +oo[— t- [0, +oo[ defined by m{r) = min{r, m{r)} 
for all r G [0,+oo[ is continuous, strictly increasing, and m{r) > Vr > 0, 
m(0) = 0. Then, for any k > 1 and for any j S {1, . . . , n^}. 



rhiUiziti)) 



1 + 



£ + 1 



< Uix), 
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so that 

Uiziti)) < ( ' + ^ ^ Uix)] . 

\e + l + ti J 

Let t G [0,t[- Then t G for some k > 1 and some j G {0, . . . , n^}. 

Moreover, by possibly reducing diam{TTk) (see Proposition 3.2), we can obtain 
*i — tir^ < ^, with 5 so small that 

uj^^{LMf5) < ^(/"fc)^^^- (27) 

Here denotes the modulus of continuity of m, when restricted to [^fc+i, c], 
L is the Lipschitz constant of U on ?7^^([/ifc+i, cr]) and Mf is the supremum 

of I/I on [/^-"^([/ifc+i, cr]) X A. Hence 

m(^(z(ti))>m(C/(z(t))i^, (28) 



which, together with (26), implies that 



Let us set 

o-_(r) = min{d(2) : U{z) > r}, o-+(r) = max{d(z) : U{z) < r}. (29) 

Notice that cj_, 0"+ : [0,+oo[— ?■ IR are continuous, strictly increasing, un- 
bounded functions such that (T_(0) = cr'^{0) = and 

\/z £U-'^{[0,a[) : a-{Uiz)) <d{z); a+ {U {z)) > d{z) . 

Moreover, it is not restrictive to replace o"_(r) with min{(T_(r), r}. Let us 
define /3 : [0, +oo[x [0, +oo[— ^ [0, +oo[ by setting 

/3(r, t) = a+o rh-' (^azHr) ^|^^) ■ (30) 

Therefore by straightforward calculations it follows that 
d{z{t)) < (3{d{x),t) VtG[0,r,[. 
It implies that, starting from any initial point x £ U~^{]0,a[), 
d{z{t)) < /3(d(x), t) yt G [0, +oo[. 
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Let us recall that, in case < +00, we mean that z{t) = z \/t > Tz, for some 



z G dC (see the footnote to Definition 1.5). By the arbitrariness of cr > 0, it is 



easy to extend the construction of /3 from U ^{[0,a]) x [0, +00 [ to the whole 
set C X [0,+oo[. 

On the one hand, this concludes the proof of part (i) of the theorem. On 



the other hand, the proof of (ii) is straightforward, since in view of (25) we 
have 

/ l{z{t),a{t))dt= lim r l{z{t),a{t))dt<^^^^U{x), 
which implies Q. 

4 Proofs of some technical results 



4.1 Proof of Proposition 3.1 



In order to prove (18) let us observe that by the definition of U, for any 
cr > 0, U^^{[0,a]) is compact. Moreover, for every 6 £]0,a] the graph of the 
restriction of the set- valued map x — )• D*U{x) to U-'^{[5,a]), namely the set 

= {{x,p) : X G U-H[5,a]), p G D*U{x)} , 

is compact. Indeed the set-valued map, x — ?■ D*U{x) is upper semicontinuous 
with compact values (see e.g. |ACj ) . Therefore the continuous function {x,p) i— >■ 
H{x,po,p) has a maximum on Ts- For every 5 g]0,(t], let us set 

m{S) = — max H{x,pQ,p) (> 0). 
(a;,p)eri 

Notice that the function rh is positive and increasing. Furthermore, it is lower 
semicontinuous. Finally, for every x G U~^{]0,a[) one has 

H{x,po,p) < —m{U{x)) \/p G D*U{x). 

The thesis is now proved by choosing, for any cr > 0, a continuous, strictly 
increasing, function m : [0,+oo[— ?• [0,+oo[ such that < m(r) < m(r) for 
every r g]0, cr] and m(0) > 0. 
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4.2 Proof of Proposition |3.2 



Let a) be a trajectory-control pair verifying conditions (a)— (f ) of Lemma 
below. Set = and, for every j G {1, . . . , n}, define 



4.1 



Set i = t". It is trivial to verify that: 
• for every j £ { 1 , . . . , n} , the path 



is a trajectory of the original system in ^ with initial condition z^ {t^ 
, corresponding to the constant control a^; 

the trajectory-control pair {z,a) : [0, t] — )• U~^{[jl, fl]) x A given by 
{z{t),a{t)) = {z^{t),a^) te[t^-\t^ (jG{l,...,n}), 



satisfies conditions (a)— (c) of Proposition 3.2 



Lemma 4.1 Let U be a (MRF), let cr > 0, and fix a selection p{x) £ D*U{x). 
Let m be defined as in Proposition \3.1\ when a is replaced with a + 2, and let 
X I— 7- a(x) £ A be a feedback lat^^erifying 

Fix e > and fl, fi such that < fi < fl < a. 

Then there exists 5 > such that, for every partition vr = (s^) of [0, +oo[ 
with diam{'K) < 6, for each x £ verifying U{x) = p,, there is a map (C, a) : 
[0,s] — )• U~^{[fi, X A verifying 

{C{s),a{s)) = {C'{s),a^) ys G W-\s^[ (j > 1), 

and a sequence (x^, x'^, . . . ) G U^^{[0, p]), where: 

(a) CMs°) = X = x^; for every j > 1, C-'(^'-'~"^) = C^~^{s^^^) = ; 



'Such a feedback exists exactly in view of Proposition 



3.1 
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(b) for every j > I, C'' '■ [s-' ^, s^] — ^ IR" is a solution of the Cauchy problem 

ds PoliC,a^) + m{U{C)y ) 

where 

= a{x^y, (32) 

(c) < +00 and s = Tl'^ 

(d) for every j > I such that s^~^ < s, one has 

U{C^{s)) - U{x^) < -'-^^ v. G W-\s^]; (33) 

(e) [/(C(s)) = fi< U{C{s)) < U{C{0)) = U{x) = fiyse [0,s[, and 

s <{e + l)U{x). (34) 

Moreover, it is possible to choose the partition vr in such a way that 

(f) s = s" for some integer n > 1, and, for every j S {1, . . . , n}, 

U{C{s^)) < U{C{s)) < U{C{s^-^) Vs G [s^-\s^[. (35) 

Proof. Fix e > and set 

g{x,a) = pol{x,a) + m{U{x)) (36) 

for all (x, a) G U~^{[fL/2,a + 1]) x A. For any continuous function (p : IR" x 
A — )• IR*^, we use M^, and co (/,{■) to denote the sup- norm and the modulus of 
continuity Qof in U-^[fi/2 , (T + 1]) X A, respectively. In case (p is scalar 
valued, let us use to denote the minimum of (j) on U~^{[fi/2,a + 1]) x A. 
Finally, since U is locally semiconcave, there exist R, p > 0, L > such that 
for aU X e Bnix, R) n U~^[fi/2,a + 1]) one has 
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U{x)-U{x)<{p,x-x)+p\x-x\'^ \fpeD*U{x), (37) 



12 

i.e 



See (|21|) for the definition of 77- 

13 



uj,p(r) = sup{\(j}(x' ,a) — (l>{x,a)\ : [x ,a'), {x,a) £ U ^ {[jl/2,a + l\)xA, \{x ,a')~{x,a)\ < r} . 



^''The inequality ( 37 1 is usually formulated with the proximal superdifferential F instead 
of dcF. However, this does not make a difference here since F — dcF as soon as F is 
locally semiconcave. 
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\p\ <L Vp G D*U{x). (38) 
Let V : K" ^ [0, 1] be a C°° (cut-off) map sucli that 

■0 = 1 on C/-i([/i/2,cj]), ^^ = on iR"\[/-i([/i/4,cj + 1]) (39) 

Let us set 



5 = mm{6i,S2,fi/2}, 

where 

. . Rrug 



(40) 



and ^2 > is any positive real number such that 

Let TT = (s-') be an arbitrary partition of [0, +oo[ such that diam(7r) < 5. 
For each x G verifying U{x) = fl, define recursively a sequence of trajectory- 
control pairs {(^ ,a^) : [s^~^, — > x A, j > 1, as follows: 

(^1(5°) = = x, a^ = a{x^y, 

for every j > 1, 

^i(sJ-i) = Ci-i(sJ-i) = rc^ a^' = a(x^'); 
for every j > 1, (^^ : [s^~^, s^] — )• IR" is a solution of the Cauchy problem 

as 5(C,a^) 

Notice that, by the continuity of the vector field and because of the cut-off 
factor -0, any trajectory C"'(-) exists globally and cannot exit the compact 
subset ?7~^([/i/4, cr -|- 1]). Let us set 

(C(s),a(s)) = (C^(s),a^) Vs G [s-'-Ss^X for every j > L 



Since, for every j > 1, one has that — ^ < (5 < 5i, by (40 ) it follows that 
|C''(s) — x^\ < R Vs G [s^~^,s^. Hence, recalling that < 1 and ijj{x^) = 1 
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as soon as G U ^{[fi/2,a + 1]), (37) and (31) imply that, for every j > I 



such that ^ < Tf (see Definition 



21), one has, Vs G [s^ ^, s^], 



C-' (r),ai) "-^ / ^ P 



< 



M 



L [si - .ci-i^ 

■9 



)) {s - s^-') + P^{s- s^-^? -is- s^-') < 



Lu.,,l,[^is^-s^-')]+P 



Ml 



f (si - - 1 



[s — s- 



Since Vs G s^], s - s^'^ < 5 < ^2, by (|4T|) it follows that 

s - s^-^ 



U{C^{s))-U{x^) < 



e + l ' 



which implies 



Uias))-Uix) = [U{C{s))-U{x^)] + --- + [UiCis^)-Uix)] < 



e + l 



(42) 



(43) 



In particular, (43) yields that U{C{s)) < fi for all s G [0,s-^]. 

Notice that < +oo. Indeed, if by contradiction = +oo, (43) held 
true for all s G [0, s^] with j arbitrarily large, i.e. (since (s^) is a partition of 
[0, +oo[), for all s > 0. Therefore one would have lim5_4.-j_oo U{C{s)) = 0, which 
is not allowed, since 



Let us set 
and 



C/(C(s))>A VsG[0,7^''[. 



s = TH< +oo), 



(44) 



n 



sup{j >\:s^ ^ < s}. 
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Let us observe that n < +00. 

Finally, notice that, because of (44), il){({s)) = 1 for every s G [0,s"']. 
Hence, for any j S {1, . . . , n}, C-'(-) is a solution of 

ds g{C,a^) 

It follows that conditions (a)— (e) are satisfied. Notice however that in general 
(f) does not hold. Indeed it may happen that s" > s. In addition, the first 
inequality of (35 ), namely U (C(s"')) < U (Cis)), may fail to be verified for some 
s e]s^-\s^[. 

In order to prove (f), it is sufficient to slightly refine the previous con- 
struction: 



In case (35) does not hold in [0,5], redefine by setting = inf{s E 
]6,6]: U{CHs))<Uiem. 

For every j > 1, choose and : [s^~^,s^~^ + (5] — t- IR" with the 
same procedure we have followed in the previous construction. In case 



(35) does not hold in [s^ + 6], set = inf{s 



+ 6] : 



U{C^{s))<U{CHs^-' + 6))}. 
The trajectory C defined by setting ('(•s) = C"'(s) Vs E [s^~^,s^[, j > 1, 



verifies (35). It remains to prove that {s =) TT = for some integer n. Begin 



with observing that U{x^^^) 
follows that 



U{C^{s^)) = U{C^is^-^ + 6)). Hence, by (42) it 



U{x^+^) - U{x^) = U[C\s^-' + <5)) - U{Q\ s^-')) < 



e + l' 



so that 



e + l 



Since j E IN", we get = for some integer n smaller than the first n such 
that n5 > {e + l){fL — fi). 



5 A remark on supersolutions 

The notion of (MRF) can be restated by replacing the strict inequality ([6]) 
with a supersolution condition. 

Preliminarly, let us recall some basic facts from nonsmooth analysis. We 
remind that we are using dcF and D*F to denote the Clarke's generalized 
gradient and the set of limiting gradients, respectively (see Definition 1.4). 
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Definition 5.1 Let 17 C H" be an open set, and let F : n ^ be a locally 
bounded function. For every x £ ^l, the set 

D-F{x) = |p G ]R" : liminf Fiv) - F{x) - {p^{y - x)) ^ ^ 
y \y — x\ 

is called the subdifferential of F. 

We recall that D^F{x) is a closed, convex (possibly empty) set. If F is 
difFerentiable at x, then D^F{x) = {VF(x)}. Moreover, when F is locally 
Lipschitz, D-F{x) C dcF{x) = coD*F{x). 

Proposition 5.1 Let U : C"^ ^ TR be a (MRF) and let po > be the constant 
for which ^ holds true. Then the strict inequality ^ can be equivalently 
replaced by the following condition: 

for every a > 0, there exists a continuous, strictly increasing function 
m = nifj : [0,+oo[— )• IR verifying m{r) > Vr > 0, such that U 
is a viscosity supersolution of equation —H{x,po,Du) — m{u) = in 
f7~-'^(]0, cr[), namely, one has 

H{x,po,D~U{x)) < -m{U{x)) Vx G [/-^(]0, (45) 

Proof. In view of Proposition 3.1, in order to show that (45) implies ^ it 
is enough to prove that, for any cj > 0, (45) implies 

H{x,po,D*U{x)) < -m{U{x)) Vx G [/-^(]0, a[) (46) 



(for the same function m as in (45)). For any x £ U ^(]0,cj[) and p G D*U{x), 
there is a sequence (x„) C U^^{]0,a[) n DLFF{U) such that 
lim„(x„, V[/(xrt)) = {x,p). Since 

D-U{x) = {VU{x)} Vx G DLFF{U), (47) 



by hypothesis ( 45 ) one has 

H (x„, Po, VU{Xn)) < -m{U{xn)), 

for each natural number n. Passing to the limit an n tends to infinity we 
get (46). The converse implication is a straightforward consequence of the 



following relations (see e.g. 

D*U{x) = D-U{x) = {yU{x)] Vx G DLFF{U); 

(48) 

D-U{x) = ^ Vx G C'=\L»/FF([/). 
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